Description:

Tech Mahindra is hiring a Data Engineer / Big Data Developer to design, construct, and sustain scalable data pipelines and distributed storage architectures. Operating out of the Chennai or Bangalore delivery centers, you will ingest and process massive volumes of structured and unstructured data from disparate corporate sources. The role focuses on transforming raw data blocks into optimized, high-quality datasets, building robust data models, troubleshooting latency bottlenecks, and establishing continuous integration data workflows to support enterprise analytics and data-driven decision-making processes.

Key Technologies:

Python, Spark, Pyspark, Data engineering

Requirements:

• Bachelor’s or Higher Degree in Computer Science, Information Technology, or a closely related quantitative field.
• 3 to 10 years of professional data engineering experience building scalable backend or big data environments.
• High proficiency in core Python scripting for writing data cleanups, pipelines, and handling abstract data frameworks.
• Hands-on development experience writing distributed analytical logic using Apache Spark and PySpark libraries to manipulate large-scale file streams.
• Practical experience interacting with enterprise cloud data warehousing solutions (such as Amazon Redshift, Google BigQuery, or Snowflake) across major public cloud platforms (AWS, Azure, or GCP).
• Familiarity configuring and scheduling tasks using Extract, Transform, Load (ETL) toolsets and modern data workflow orchestration tools like Apache Airflow, Apache NiFi, or Talend.
• Robust foundational command over SQL scripting alongside deep structural database management system (DBMS) experience handling relational platforms (MySQL, PostgreSQL) or NoSQL data clusters (MongoDB).
• Build and run strict testing and structural validation processes to safeguard data integrity and quality across production storage components. Maintain technical maps, architecture layouts, and data lineage documentation.
• Sharp analytical troubleshooting capacities, exceptional problem-solving habits under abstract system contexts, and clear communication or collaboration skills to interface cleanly with data science and analytics squads.